Toss that BOSSbase, Alice!

Steganographic schemes for digital images are routinely designed and benchmarked based on feedback obtained on the standard image set called BOSSbase 1.01. While standardized image sets are important for advancing the field, relying on results from a single source may not provide fair benchmarking and may even lead to designs that are overoptimized and highly suboptimal on other image sources. In this paper, we investigate four modern steganographic schemes for the spatial domain, WOW, S-UNIWARD, HILL, and MiPOD on two more versions of BOSSbase. We observed that with their default settings, the mutual ranking and detectability of all four embedding algorithms can dramatically change across the three image sources. For example, in a version of BOSSbase whose images were cropped instead of resized, all four schemes exhibit almost the same empirical security when steganalyzed with the spatial rich model (SRM). On the other hand, in decompressed JPEG images, WOW is the most secure embedding algorithm out of the four, and this stays true irrespectively of the JPEG quality factor when steganalyzing with both SRM and maxSRM. The empirical security of all four schemes can be increased by optimizing the parameters for each source. This is especially true for decompressed JPEGs. However, the ranking of stego schemes still varies depending on the source. Through this work, we strive to make the community aware of the fact that empirical security of steganographic algorithms is not absolute but needs to be considered within a given environment, which includes the cover source.